Second-order Stein: SURE for SURE and other applications in high-dimensional inference

Stein’s formula states that a random variable of the form z⊤f(z)−divf(z) is mean-zero for all functions f with integrable gradient. Here, divf divergence function and z standard normal vector. This paper aims to propose second-order Stein characterize variance such variables f(z) square gradient, demonstrate usefulness this in various applications. In Gaussian sequence model, remarkable consequence Unbiased Risk Estimate (SURE), an unbiased estimate mean squared risk almost any given estimator μˆ unknown A first application SURE itself (SURE SURE): providing information about distance between estimation error μˆ. has simple as data applicable example, Lasso Elastic Net. addition SURE, following statistical applications are developed: (1) upper bounds on when target error; (2) confidence regions based using formula; (3) oracle inequalities satisfied by SURE-tuned estimates under mild Lipschtiz assumption; (4) bound size model selected Lasso, more generally empirical degrees-of-freedom convex penalized estimators; (5) explicit expressions Net; (6) linear general semiparametric scheme de-bias differentiable initial inference low-dimensional projection regression coefficient vector, characterization after debiasing; (7) accuracy analysis Monte Carlo approximate f:Rn→Rn.